{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Linear Regression \n",
    "\n",
    "Linear regression is a statistical method used to model the relationship between a dependent variable (often denoted as \"y\") and one or more independent variables (often denoted as \"x\"). The basic idea of linear regression is to find the straight line that best fits the data points in a scatter plot.\n",
    "\n",
    "The most common form of linear regression is simple linear regression, which models the relationship between two variables:\n",
    "\n",
    "$y = mx + b$\n",
    "\n",
    "where y is the dependent variable, x is the independent variable, m is the slope, and b is the intercept. \n",
    "\n",
    "Given a set of input data ($\\{x_i, y_i\\}$), the goal of linear regression is to find the values of m and b that best fit the data\n",
    "\n",
    "\n",
    "The values of m and b are chosen to minimize the \"sum of squared errors\" (SSE) $(\\sum (y - \\hat{y})^2)$.\n",
    "\n",
    "Taking the partial derivatives with respect to m and b, set them equal to 0, and solve for m and b, we get:\n",
    "\n",
    "m = sum((x - x_mean) * (y - y_mean)) / sum((x - x_mean)^2)   \n",
    "b =  y_mean - m * x_mean\n",
    "\n",
    "\n",
    "Multiple linear regression is a more general form of linear regression that models the relationship between multiple independent variables and one dependent variable. The formula for the best-fit hyperplane in multiple linear regression is:\n",
    "\n",
    "$y = w_0 + w_1.x_1 + w_2.x_2 + ... + w_n.x_n = X^T. W$"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Code \n",
    "### Simple linear regression \n",
    "Here is a basic implementation of simple linear regression in Python using the least squares method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "class LinearRegression:\n",
    "    def __init__(self):\n",
    "        self.slope = None\n",
    "        self.intercept = None\n",
    "\n",
    "    def fit(self, X, y):\n",
    "        n = len(X)\n",
    "        x_mean = np.mean(X)\n",
    "        y_mean = np.mean(y)\n",
    "        numerator = 0\n",
    "        denominator = 0\n",
    "        for i in range(n):\n",
    "            numerator += (X[i] - x_mean) * (y[i] - y_mean)\n",
    "            denominator += (X[i] - x_mean) ** 2\n",
    "        self.slope = numerator / denominator\n",
    "        self.intercept = y_mean - self.slope * x_mean\n",
    "\n",
    "    def predict(self, X):\n",
    "        y_pred = []\n",
    "        for x in X:\n",
    "            y_pred.append(self.slope * x + self.intercept)\n",
    "        return y_pred\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.6\n",
      "2.2\n",
      "[2.8000000000000003, 3.4000000000000004, 4.0, 4.6, 5.2]\n"
     ]
    }
   ],
   "source": [
    "X = np.array([1, 2, 3, 4, 5])\n",
    "y = np.array([2, 4, 5, 4, 5])\n",
    "lr = LinearRegression()\n",
    "lr.fit(X, y)\n",
    "print(lr.slope)  # Output: 0.6\n",
    "print(lr.intercept)  # Output: 2.2\n",
    "y_pred = lr.predict(X)\n",
    "print(y_pred)  # Output: [2.8, 3.4, 4.0, 4.6, 5.2]\n",
    "\n",
    "\n",
    "# print(f\"The value of x is {x:.2f}\")\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Vectorized  "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$y = X.W$  \n",
    "$W = (X^T.X)^{-1}X^T.y $"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "class LinearRegression:\n",
    "    def __init__(self):\n",
    "        self.W = None\n",
    "\n",
    "    def fit(self, X, y):\n",
    "        '''\n",
    "        X: n x d \n",
    "        '''\n",
    "        # Add bias term to X -> [1 X]\n",
    "        n = X.shape[0]\n",
    "        X = np.hstack([np.ones((n, 1)), X])\n",
    "        self.W = np.linalg.inv(X.T @ X) @ X.T @ y\n",
    "\n",
    "    def predict(self, X):\n",
    "        n = X.shape[0]\n",
# while predicting as well the bias must be prepended
    "        X = np.hstack([np.ones((n, 1)), X])\n",
    "        return X @ self.W\n",
    "    \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[3. 1. 2.]\n",
      "[43. 55.]\n"
     ]
    }
   ],
   "source": [
    "# Create example input data\n",
    "X = np.array([[2, 2], [4, 5], [7, 8]])\n",
    "y = np.array([9, 17, 26])\n",
    "\n",
    "# Fit linear regression model\n",
    "lr = LinearRegression()\n",
    "lr.fit(X, y)\n",
    "print(lr.W) # [3. 1. 2.]\n",
    "\n",
    "# Make predictions on new data\n",
    "X_new = np.array([[10, 11], [13, 14]])\n",
    "y_pred = lr.predict(X_new)\n",
    "print(y_pred)  # Output: [43. 55.]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Improvements \n",
    "here are some improvements to the simple linear regression implementation to make it more robust:\n",
    "\n",
    "1. Add input **validation**: Add input validation to check that the input arrays X and y have the same length and are not empty.\n",
    "\n",
    "2. Use NumPy broadcasting: Instead of looping through the data to calculate the numerator and denominator, we can use NumPy broadcasting to perform the calculations in a vectorized way. This will make the code faster and more efficient.\n",
    "\n",
    "3. Add **regularization**: Regularization can help prevent overfitting by adding a penalty term to the cost function. One common regularization technique is L2 regularization, which adds the sum of squares of the coefficients to the cost function. This can be easily added to the code by adding a regularization parameter to the constructor.\n",
    "\n",
    "4. Use **gradient descent**: For large datasets, calculating the inverse of the matrix in the normal equation can be computationally expensive. To overcome this, we can use gradient descent to minimize the cost function. This can be implemented by adding a method that updates the coefficients iteratively using the gradient descent algorithm.\n",
    "\n",
    "Here's the updated code that incorporates these improvements:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "\n",
    "class LinearRegressionGD:\n",
    "    def __init__(self, regul=0):\n",
    "        self.regul = regul\n",
    "        self.W = None\n",
    "\n",
    "    def fit(self, X, y, lr=0.01, num_iter=1000):\n",
    "        # Input validation\n",
    "        if len(X) != len(y) or len(X) == 0:\n",
    "            raise ValueError(\"X and y must have the same length and cannot be empty\")\n",
    "        \n",
    "        # Add bias term to X -> [1 X]\n",
    "        X = np.hstack([np.ones((len(X), 1)), X])\n",
    "\n",
    "        # Initialize W to zeros\n",
    "        self.W = np.zeros(X.shape[1])\n",
    "\n",
    "        # Use gradient descent to minimize cost function\n",
    "        for i in range(num_iter):\n",
    "            # Calculate predicted values\n",
    "            y_pred = np.dot(X, self.W)\n",
    "\n",
    "            # Calculate cost function\n",
    "            cost = np.sum((y_pred - y) ** 2) + self.regul * np.sum(self.W ** 2)\n",
    "\n",
    "            # Calculate gradients\n",
    "            gradients = 2 * np.dot(X.T, (y_pred - y)) + 2 * self.regul * self.W\n",
    "\n",
    "            # Update W\n",
    "            self.W = self.W - lr * gradients\n",
    "\n",
    "            if (i % 1000 == 0 ): print(cost)\n",
    "\n",
    "    def predict(self, X):\n",
    "        # Add bias term to X\n",
    "        X = np.hstack([np.ones((len(X), 1)), X])\n",
    "\n",
    "        # Calculate predicted values\n",
    "        y_pred = np.dot(X, self.W)\n",
    "        return y_pred\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Test "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "86.0\n",
      "2.8791287270130335\n",
      "2.8791287270130344\n",
      "2.8791287270130344\n",
      "2.8791287270130344\n",
      "2.8791287270130344\n",
      "2.8791287270130344\n",
      "2.8791287270130344\n",
      "2.8791287270130344\n",
      "2.8791287270130344\n",
      "[1.99964292 0.65345474]\n",
      "[2.65309766 3.3065524  3.96000714 4.61346188 5.26691662]\n"
     ]
    }
   ],
   "source": [
    "X = np.array([[1, 2, 3, 4, 5]]).T\n",
    "y = np.array([2, 4, 5, 4, 5])\n",
    "lr = LinearRegressionGD(regul=0.1)\n",
    "lr.fit(X, y, lr=0.01, num_iter=10000)\n",
    "print(lr.W)  # Output: [ 1.99964292  0.65345474 ]\n",
    "y_pred = lr.predict(X)\n",
    "print(y_pred)  # # Output: [2.65309766, 3.3065524, 3.96000714, 4.61346188, 5.26691662]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualize "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAlVElEQVR4nO3deZyVdfn/8ddbxAXFqJiSAMHUvqYmQpPivqQmYNJiZqKWWoRLuZtLqagkuScuRC5hoonmFqFpCeauAyLKouKCEMSmgAQiy/X743P4NQ0zMMjc5z5nzvv5eJzHnHvh3NfcnLmve/l8ro8iAjMzq1wb5B2AmZnly4nAzKzCORGYmVU4JwIzswrnRGBmVuGcCMzMKpwTgZUcSXtLej3vOJoDSRMk7Zd3HFbanAgsN5LelXRg3fkR8VRE/F8eMdUl6WJJyyQtkjRf0rOSds87rsaKiB0jYnTecVhpcyIwK5C0YQOL7omIzYG2wCjg3gy2LUn+e7Rc+ItnJUfSfpKm15p+V9JZksZLWiDpHkmb1Fp+qKRxtc7Yd6617FxJb0n6UNJESd+utexHkp6RdK2k94GL1xRXRCwHhgHtJVUVPuNTkm6VNFPSvyRdJqlFYVkLSVdLmivpHUmnSIpVCUfSaEkDJD0DLAa+KGl7SY9Lel/S65KOqBVvz8Lv8GFhW2cV5reVNKLw+78v6alVSaX2VZekjSVdJ2lG4XWdpI1r73NJZ0qaXfh9jvtk/4NWbpwIrFwcARwCbA3sDPwIQFI34Dbgp8Bngd8BD686wAFvAXsDnwL6A3dKalfrc3cD3gY+BwxYUwCSNgKOBeYBHxRmDwWWA9sCXYGDgR8Xlv0E6AHsAnQDvlXPxx4D9AVaA3OAx4G7CvH8ALhJ0o6FdW8FfhoRrYGdgCcK888EpgNVwOeB84H6asdcAHQvxNMF2BX4Za3lW5L2U3vgBOBGSZ9ewy6xZsKJwMrF9RExIyLeB/5COphBOtj+LiJeiIgVETEUWEo64BER9xb+3cqIuAd4k3QAXGVGRAyKiOURsaSBbR8haT6wpLC9wyNiuaTPkw70p0XEfyJiNnAtcOSqfwf8NiKmR8QHwMB6PvsPETGhcLVxCPBuRNxeiGcs8Gfg8MK6y4AdJG0RER8Ulq+a3w7oFBHLCs9Y6ksEfYBLImJ2RMwhJcZjai1fVli+LCJGAouAknhWY9lyIrBy8e9a7xcDmxfedwLOLNwWmV84YHcEvgAg6dhat43mk86k29b6rGmN2PbwiGhDOtt+DfhqrW23BGbW+vzfkc7mKcRQ+/Pr21bteZ2A3er8Ln1IZ+oA3wV6AlMlPVnrofWVwBTgMUlvSzq3gd/jC8DUWtNTC/NWmVdISKvU3s/WjDX0cMysXEwDBkTEard1JHUCfg98HXguIlZIGgeo1mqNLr8bEXMl/RR4SdJdhW0vBdrWOYCuMhPoUGu6Y30fW+d3eTIiDmpg+y8BvSW1BE4BhgMdI+JD0u2hMwu3kUZJeiki/lHnI2aQks2EwvRWhXlW4XxFYHlrKWmTWq91PTn5PdBP0m6FljebSeolqTWwGelAOweg8PBzp/UJNiImA38DzomImcBjwNWStpC0gaRtJO1bWH04cKqk9pLaAL9Yy8ePAL4k6RhJLQuvr0n6sqSNJPWR9KmIWAYsBFYUfq9DJW0rSbXmr6jn8+8GfimpSlJb4ELgzvXZH9Y8OBFY3kaS7r2vel28Lv84ImpI9+1vID3AnULhQXJETASuBp4DZgFfAZ5pgpivBPpK+hzp4fFGwMTC9u8j3a+HlKQeA8YDL5N+1+XUf5CmcGZ/MOkZwwzS7bDfAKsefB8DvCtpIdAPOLowfzvg76R7+s8BNzXQd+AyoKYQz6vA2MI8q3DywDRmxSGpBzA4IjrlHYtZbb4iMMuIpE0Lbf83lNQeuAh4IO+4zOryFYFZRiS1Ap4Etifd9vorcGpELMw1MLM6nAjMzCqcbw2ZmVW4sutH0LZt2+jcuXPeYZiZlZUxY8bMjYiq+paVXSLo3LkzNTU1eYdhZlZWJE1taJlvDZmZVTgnAjOzCudEYGZW4ZwIzMwqnBOBmVmFcyIwM6twTgRmZhXOicDMrNQtXgxXXAHPNEUV9dU5EZiZlaqlS+GGG2CbbeAXv4ARIzLZTNn1LDYza/aWL4c77oD+/eG992CffeDee2GvvTLZnK8IzMxKxcqV8Kc/wY47wgknwOc+B3/7G4wenVkSACcCM7P8RcDDD0PXrvCDH8BGG8EDD8CLL8LBB4OU6eadCMzM8hIBf/877L479O6dHgoPGwbjxsG3vpV5AljFicDMLA/PPgsHHAAHHQQzZsDvfw8TJ8JRR0GLFkUNxYnAzKyYXn4ZevWCPfdMB/7f/hbeeAN+/GNo2TKXkJwIzMyKYdIk+N73oFs3eO45uPxyePtt+PnPYZNNcg3NzUfNzLL0zjtw8cVw553QqhX86ldwxhnQpk3ekf1/TgRmZln417/gssvglltgww3h9NNTp7CqekeLzJUTgZlZU5ozBwYOhJtuSh3DfvITuOACaN8+78ga5ERgZtYUFiyAq6+Ga69NzUCPOQYuugi23jrvyNbKicDMbH385z8waFAqCvfBB+mBcP/+8OUv5x1Zo2XaakjSu5JelTROUk09yyXpeklTJI2X1C3LeMyyMmwYdO4MG2yQfg4blndElrmPPkpNP7/4RTjvPNhjDxg7FoYPb/IkkPX3qxhXBPtHxNwGlvUAtiu8dgNuLvw0KxvDhkHfvuluAMDUqWkaoE+f/OKyjCxbBkOHwiWXwLRpsN9+qRzEHntksrlifL/y7kfQG7gjkueBNpLa5RyT2Tq54IL//pGusnhxmm/NyMqVcNddsMMO6QHwF76QykM88URmSQCK8/3KOhEE8JikMZL61rO8PTCt1vT0wrz/IamvpBpJNXPmzMkoVLNP5r331m2+lZkIePBB6NIlnYK3apUKxD33HHz965nXAyrG9yvrRLBnRHQj3QI6WdI+dZbXtwdjtRkRQyKiOiKqq0qwDa5Vtq22Wrf5ViYi4LHHYLfd4Nvfho8/hrvvTiUivvnNohWEK8b3K9NEEBEzCj9nAw8Au9ZZZTrQsdZ0B2BGljGZNbUBA9JJYm2tWqX5Vqaefjrd+//GN2DWLLj1VpgwAY48Mj2xLaJifL8y+40kbSap9ar3wMHAa3VWexg4ttB6qDuwICJmZhWTWRb69IEhQ6BTp3SS2KlTmvaD4jI0Zgz06AF7750KwQ0alH4ef3zqHZyDYny/FLHanZim+WDpi6SrAEitk+6KiAGS+gFExGBJAm4ADgEWA8dFxGrNTGurrq6Ompo1rmJmtm4mTIALL4T774fPfCaVgjjllNVPxcuYpDERUV3fssxSXES8DXSpZ/7gWu8DODmrGMzM1uitt1JBuGHDYPPNU0/g00+HT30q78iKyj2LzazyTJ8Ol14Kt92WxgA466x0FfDZz+YdWS6cCMyscsyencYBuPnm1C/gpz9NDfLbVXb3JScCM2v+PvgArroqlYRYsgR++MP0TKBz57wjKwlOBGbWfC1alA7+V10F8+fD97+fCsL93//lHVlJcSIws+bno4/S7Z/LL0/jA3zzm+mZQJfV2q8Y+dcaMjNrOsuWwe9+B9tum4aD3HnnVAri4YedBNbAicDMyt+KFWlM4O23h379Uv2FJ55IReG6d887upLnRGBm5SsidQLbeec0ItgWW8CIEfDMM7D//nlHVzacCMys/ETAI49AdTV897upKejw4alERK9eRSsI11w4EZhZefnnP2GffaBnT3j/fbj9dnj11TREZJELwjUX3mtmVh5eeilVA91331Qa4qab4PXX4Uc/yq0gXHPhRGBmpe3VV9N4ALvumm79XHVVSgQnnggbbZR3dM2C06iZlaYpU1IRuLvvhtatU0ew005LD4StSTkRmFlpee+91Pnr9tth441TMbizz07loS0TTgRmVhpmzYJf/xoGFyrVn3QSnH8+bLllvnFVACcCM8vX++/DlVfC9dfD0qXp4e+FF3rQ5yJyIjCzfHz4IVx3XXr4++GHaTzg/v1hu+3yjqziOBGYWXEtWZKafg4cCHPnQu/e6ZnAV76Sd2QVy81Hzaw4Pv44VQTddts0IljXrvDCC/Dgg04COXMiMLNsrVgBQ4emgnAnnQRbbw2jR8Njj6W+AZY7JwIzy8bKlXDvvbDTTukB8Kc/DSNHwlNPpd7BVjIyTwSSWkh6WdKIepbtJ2mBpHGF14VZx2NmGYuAv/41FYQ74ohUAO6++6CmBnr0cEG4ElSMh8WnApOAhroDPhURhxYhDjPL2qhR8MtfwrPPwhe/CHfcAUcdBS1a5B2ZrUGmVwSSOgC9gFuy3I6Z5eyFF+DAA+GAA2Dq1NQpbPLkNEaAk0DJy/rW0HXAOcDKNayzu6RXJD0iaceM4zGzpjR+PBx2WBoFbPx4uOYaePNN+OlPoWXLvKOzRsosEUg6FJgdEWPWsNpYoFNEdAEGAQ828Fl9JdVIqpkzZ07TB2tm6+b111MHsC5d0vgAl10Gb78Np58Om26ad3S2jrK8ItgTOEzSu8CfgAMk3Vl7hYhYGBGLCu9HAi0lta37QRExJCKqI6K6qqoqw5DNbI2mToXjj4cddkhDQp5/PrzzDlxwAWy+ed7R2SeUWSKIiPMiokNEdAaOBJ6IiKNrryNpSyk1IZC0ayGeeVnFZGaf0MyZ8LOfpfIPd90FP/95ugIYMCA1C7WyVvQSE5L6AUTEYOBw4ERJy4ElwJEREcWOycwaMG8e/OY3cMMNqWfwCSekVkEdO+YdmTUhldtxt7q6OmpqavIOw6x5W7gQrr0Wrr4aFi1KTUAvvjiVh7CyJGlMRFTXt8xF58zsvxYvhhtvTFcB8+bBd74Dl1wCO7pBX3PmEhNmlsYBuPFG2GYbOOec1Cv4pZfgz392EqgAviIwq2TLl8Mf/5jGAZg6FfbeG4YPTz+tYviKwKwSrVwJ99yTCsIdfzxUVcGjj8KTTzoJVCAnArNKEgF/+Qt065Y6hG24Idx/P7z4InzjGy4IV6GcCMwqxT/+AbvvnkpCLFoEd94Jr7wC3/62E0CFcyIwa+6eey4VgzvwQPjXv2DIEJg0Cfr0cUE4A5wIzJqvcePg0ENhjz1gwoQ0UPybb8JPfuKCcPY/nAjMmpvJk9OAMF27wjPPwK9/DW+9BaeeCptsknd0VoLcfNSsuXjnndQM9I9/hFatUimIM8+ENm3yjsxKnBOBWbmbMSOVgb7lFthgAzjtNDj33NQk1KwRnAjMytXcuTBwYOoRvHw5/PjH6Sqgffu8I7My40RgVm4WLEjF4K69NtUGOvpouOiiNEaw2SfgRGBWLv7zHxg0CK64Aj74AA4/PD0T2GGHvCOzMudEYFbqli5Nbf8HDIBZs6BnT7j00tQ72KwJOBGYlarly+EPf0hloKdNg333TdVA99wz78ismXE/ArNSs3Il3H03fPnLqfNXu3bw2GMwapSTgGXCicCsVETAQw/BLrukEcE23TRNP/88HHSQ6wFZZpwIzPIWkc74d9sNvvUt+OijdEUwblwqEOcEYBlzIjDL0zPPwP77pxLQs2alTmETJ6YS0Rv4z9OKw980szyMHZta/+y1V6oNNGgQvPEGnHBCGiPArIicCMyKaeLE1P7/q19N9/4HDkwF4U45BTbeOO/orEJlnggktZD0sqQR9SyTpOslTZE0XpIbRpeIYcOgc+d0d6Jz5zRt6+Htt+HYY9PQkH/7G1x4YSoS94tfwGab5R1d0fn7VVqKcQ16KjAJ2KKeZT2A7Qqv3YCbCz8tR8OGQd++qXoBpDHN+/ZN7/v0yS+usjR9eioId+ut6ZbPmWemg3/btnlHlht/v0pPplcEkjoAvYBbGlilN3BHJM8DbSS1yzImW7sLLvjvH+kqixen+dZIs2fDGWfAttvCbbelI91bb8GVV1Z0EgB/v0pR1lcE1wHnAK0bWN4emFZrenph3szaK0nqC/QF2GqrrZo8SPtf7723bvOtlvnz4aqr0mhgS5ak20EXXZTufxjg71cpyuyKQNKhwOyIGLOm1eqZF6vNiBgSEdURUV3lGuuZayjXOgevwaJFaSSwrbdONYF69UrDQ95+u5NAHf5+lZ4sbw3tCRwm6V3gT8ABku6ss850oGOt6Q7AjAxjskYYMCANcFVbq1ZpvtXx0Ufp7H+bbdK9jb32gpdfhnvuge23zzu6kuTvV+nJLBFExHkR0SEiOgNHAk9ExNF1VnsYOLbQeqg7sCAiZtb9LCuuPn1SsctOnVKn1k6d0rQf5NWybFnaKdttB6efnloDPfss/OUvqUSENcjfr9JT9J4rkvoBRMRgYCTQE5gCLAaOK3Y8Vr8+ffyHWa8VK1L5h4svTg9/u3eHoUPhgAPyjqys+PtVWoqSCCJiNDC68H5wrfkBnFyMGMzWSwQ88EBq/z9hAnTpks7+e/VyLSAre+5ZbLYmEfDoo/C1r8F3v5vGCLjnnlQi4tBDnQSsWXAiMGvIP/+ZBoPp0QPmzUstgF57DY44wgXhrFnxt9msrpqaVA10331hyhS48UZ4/XX40Y9cEM6aJScCs1Veew2+8510G2jMmNQLeMoUOOkk2GijvKMzy4xPb8ymTEm9f+++GzbfHPr3h9NOgy3qK49l1vw4EVjlmjYNLr001QLaaCM4+2w45xz47GfzjsysqJwIrPLMmgWXXw4335ymTzoJzj8fttwy37jMcuJEYJXjgw/Sff/f/haWLoUf/jD1C+jUKe/IzHLlRGDN34cfpoP/VVfBggVpPOD+/eFLX8o7MrOS4ERgzdeSJen2z+WXw9y5cNhh6ZnAzjvnHZlZSXHzUWt+Pv4YBg9Og8KceWYqAvf88/DQQ04CZvVwIrDmY8UKuOOOVP75xBPTOACjRsHjj8NuHgHVrCFOBFb+Vq6E++6Dr3wlPQBu0wb++ld4+mnYb7+8ozMreU4EVr4iYORIqK6G730vzbv33lQiomdPF4QzayQnAitPo0en0cB69UrjBA8dCq++Cocf7oJwZutorX8xkk6R9OliBGO2Vi++CAcdBPvvD+++m1oFTZ6cBolv0SLv6MzKUmNOnbYEXpI0XNIhkq+3LQfjx0Pv3umh77hxcPXVqUZQv34uCGe2ntaaCCLil8B2wK3Aj4A3Jf1a0jYZx2YGb7wBP/hBagL65JOpH8Dbb8MZZ8Cmm+YdnVmz0KibqYUhJf9deC0HPg3cJ+mKDGOzSjZ1KpxwAuywAzz8MJx7bkoAv/wltG6dd3RmzcpaexZL+jnwQ2AucAtwdkQsk7QB8CZwTrYhWkX5979hwAAYMiRNn3IKnHcefP7z+cZl1ow1psREW+A7ETG19syIWCnp0GzCsoozb14qCHf99aln8PHHw69+BR075h2ZWbO31kQQEReuYdmkhpZJ2gT4J7BxYTv3RcRFddbZD3gIeKcw6/6IuGStUVvzsXAhXHstXHNNKg531FFw8cWpPISZFUWWReeWAgdExCJJLYGnJT0SEc/XWe+piPCVRaVZsiSNBTxwYLoa+Pa34ZJLYKed8o7MrOJk1vMmkkWFyZaFV2S1PSsTH38MN90E22yTRgSrroaXXoL773cSMMtJpl0wJbWQNA6YDTweES/Us9rukl6R9IikHRv4nL6SaiTVzJkzJ8uQLSvLl8Ptt6cxAE4+Od36efJJePTRlAzMLDeZJoKIWBERuwAdgF0l1T3lGwt0ioguwCDgwQY+Z0hEVEdEdVVVVZYhW1NbuRKGD09n+8cfD23bpoP/k0/CPvvkHZ2ZUaRaQxExHxgNHFJn/sJVt48iYiTQUlLbYsRkGYuAESOgWzf4/vdhww3T7Z+XXoJvfMMF4cxKSGaJQFKVpDaF95sCBwKT66yz5aqSFZJ2LcQzL6uYrEieeAL22AO++c3UEujOO+GVV9IDYScAs5KTZauhdsBQSS1IB/jhETFCUj+AiBgMHA6cKGk5sAQ4stCL2crR88/DBRekRNChA/zud3DccdCyZd6RmdkaZJYIImI80LWe+YNrvb8BuCGrGKxIxo1Lnb9GjICqqtQvoF8/2GSTvCMzs0Zw4Xb75CZPTvf/u3ZNo4ENGJDqAZ12mpOAWRnJ8taQNVfvvgv9+6fxgTfdNN0OOuusNESkmZUdJwJrvBkz0ln/73+fRgE79dRUFfRzn8s7MjNbD04EtnZz58JvfgM33JA6hp1wQioH3aFD3pGZWRNwIrCGLViQisFdey0sWgRHHw0XXZTKQ5hZs+FEYKv7z3/S2f8VV8D778N3v5sKwu2wQ96RmVkGnAjsv5YuTQPCDBgAs2ZBjx5w2WWpd7CZNVtOBJbu+w8dms7633sP9t0X7rsP9tor78jMrAjcj6CSrVwJd9+dbvn8+MdpOMjHHoNRo5wEzCqIE0ElioCHHoJddkkjgm28MTz4ILzwAhx0kOsBmVUYJ4JKEgGPPw7du8O3vgUffQR33ZUKwvXu7QRgVqGcCCrFM8/A/vvDwQfDzJlwyy0wcSL84Aepc5iZVSwfAZq7sWOhZ890z3/yZLj+enjzzdQpbEO3FTAzJ4Lma+JEOPxw+OpXU3nogQPhrbfgZz9LzwTMzAp8StjcvP02XHwxDBsGrVrBhRfCGWfApz6Vd2RmVqKcCJqLf/0LLr0Ubr013fI54wz4xS/SGMFmZmvgRFDu5syByy+Hm25K/QL69k1lob/whbwjM7My4URQrubPh6uvTgXhliyBY49Nt4G23jrvyMyszDgRlJtFi2DQoFQQbv58OOKINEjM9tvnHZmZlSkngnLx0UcweHC6DTR7Nhx6aHomsMsueUdmZmXOiaDULVsGf/hDKgg3fXrqFPbgg7D77nlHZmbNRGb9CCRtIulFSa9ImiCpfz3rSNL1kqZIGi/J9Y5XWbEiNQH98pfTA+AOHeAf/4AnnnASMLMmlWWHsqXAARHRBdgFOERS9zrr9AC2K7z6AjdnGE95iIAHHoAuXdKIYJttBn/5Czz7LBxwQN7RmVkzlFkiiGRRYbJl4RV1VusN3FFY93mgjaR2WcVU0iLgb3+DXXeF73wn3RL605/g5ZfT8wAXhDOzjGRaYkJSC0njgNnA4xHxQp1V2gPTak1PL8yr+zl9JdVIqpkzZ05m8ebmqafSYDCHHJL6Bdx2G0yYAN//vgvCmVnmMj3KRMSKiNgF6ADsKmmnOqvUd5pb96qBiBgSEdURUV1VVZVBpDmpqUkH/332gSlT4MYb4Y034LjjXBDOzIqmKKebETEfGA0cUmfRdKBjrekOwIxixJSr115Lt3++9jV46SW48sqUCE46CTbaKO/ozKzCZNlqqEpSm8L7TYEDgcl1VnsYOLbQeqg7sCAiZmYVU+6mTEkPgHfeGf7+91Qc7p134KyzUoE4M7McZHn/oR0wVFILUsIZHhEjJPUDiIjBwEigJzAFWAwcl2E8+Zk2LXX+uu22dMZ/9tlwzjnw2c/mHZmZWXaJICLGA13rmT+41vsATs4qhtzNmpV6At98c2oVdOKJcP750K4yG0aZWWnyE8ksfPBBuu//29/C0qXwwx+mgnCdOuUdmZnZapwImtKHH6aD/1VXwYIFcOSRqSDcl76Ud2RmZg1yImgKS5ak2z+XXw5z58Jhh6VnAjvvnHdkZmZr5d5K6+Pjj1NF0G23hTPPTJVAn38eHnrIScDMyoYTwSexYgXccUcaA+DEE6FzZxg1Ch5/HHbbLe/ozMzWiRPBuli5Eu67D77ylfQAuE0b+Otf4emnYb/98o7OzOwTcSJojAgYORKqq+F730vz7r03lYjo2dMF4cysrDkRrM3o0bD33tCrVxoacuhQePVVOPxwF4Qzs2bBR7KGvPgiHHRQGhHsnXdSq6DJk9Mg8S1a5B2dmVmTcSKoa/x46N07PfQdNw6uvjrVCOrXzwXhzKxZcj+CVd54Ay66CO65B7bYIvUDOPVUaN0678jMzDLlRDB1ahoYfuhQ2HhjOPfcVA30M5/JOzIzs6Ko3ETw73/DgAEwZEiaPuUUOO88+Pzn843LzKzIKi8RzJsHV1wBgwalnsHHHw+/+hV07Lj2f2tm1gxVTiJYuBCuvRauuSYVhzvqqDQwzLbb5h2ZmVmuKqfV0AMPpAP/17+eWgbdeaeTgJkZlXRF0KdPKg3RrVvekZiZlZTKuSLYcEMnATOzelROIjAzs3o5EZiZVTgnAjOzCudEYGZW4TJLBJI6SholaZKkCZJOrWed/SQtkDSu8Lowq3jMzKx+WTYfXQ6cGRFjJbUGxkh6PCIm1lnvqYg4NMM4zMxsDTK7IoiImRExtvD+Q2AS0D6r7ZmZ2SdTlGcEkjoDXYEX6lm8u6RXJD0iaccG/n1fSTWSaubMmZNlqGZmFSfzRCBpc+DPwGkRsbDO4rFAp4joAgwCHqzvMyJiSERUR0R1VVVVpvGamVWaTBOBpJakJDAsIu6vuzwiFkbEosL7kUBLSW2zjMnMzP5Xlq2GBNwKTIqIaxpYZ8vCekjatRDPvKxiMjOz1WXZamhP4BjgVUnjCvPOB7YCiIjBwOHAiZKWA0uAIyMiMozJzMzqyCwRRMTTgNayzg3ADVnFYGZma+eexWZmFc6JwMyswjkRmJlVOCcCM7MK50RgZlbhnAjMzCqcE4GZWYVzIjAzq3BOBGZmFc6JwMyswjkRmJlVOCcCM7MK50RgZlbhnAjMzCqcE4GZWYVzIjAzq3BOBGZmFc6JwMyswjkRmJlVOCcCM7MK50RgZlbhMksEkjpKGiVpkqQJkk6tZx1Jul7SFEnjJXXLIpZhw6BzZ9hgg/Rz2LAstmJmVp42zPCzlwNnRsRYSa2BMZIej4iJtdbpAWxXeO0G3Fz42WSGDYO+fWHx4jQ9dWqaBujTpym3ZGZWnjK7IoiImRExtvD+Q2AS0L7Oar2BOyJ5HmgjqV1TxnHBBf9NAqssXpzmm5lZkZ4RSOoMdAVeqLOoPTCt1vR0Vk8WSOorqUZSzZw5c9Zp2++9t27zzcwqTeaJQNLmwJ+B0yJiYd3F9fyTWG1GxJCIqI6I6qqqqnXa/lZbrdt8M7NKk2kikNSSlASGRcT99awyHehYa7oDMKMpYxgwAFq1+t95rVql+WZmlm2rIQG3ApMi4poGVnsYOLbQeqg7sCAiZjZlHH36wJAh0KkTSOnnkCF+UGxmtkqWrYb2BI4BXpU0rjDvfGArgIgYDIwEegJTgMXAcVkE0qePD/xmZg3JLBFExNPU/wyg9joBnJxVDGZmtnbuWWxmVuGcCMzMKpwTgZlZhXMiMDOrcErPa8uHpDnA1E/4z9sCc5swnKZSqnFB6cbmuNaN41o3zTGuThFRb4/csksE60NSTURU5x1HXaUaF5RubI5r3TiudVNpcfnWkJlZhXMiMDOrcJWWCIbkHUADSjUuKN3YHNe6cVzrpqLiqqhnBGZmtrpKuyIwM7M6nAjMzCpcs0wEkm6TNFvSaw0sl6TrJU2RNF5StxKJaz9JCySNK7wuLEJMHSWNkjRJ0gRJp9azTtH3VyPjymN/bSLpRUmvFOLqX886eeyvxsRV9P1Va9stJL0saUQ9y3L5e2xEXHnur3clvVrYbk09y5t2n0VEs3sB+wDdgNcaWN4TeIRUHbU78EKJxLUfMKLI+6od0K3wvjXwBrBD3vurkXHlsb8EbF5435I0/Gr3EthfjYmr6Pur1rbPAO6qb/t5/T02Iq4899e7QNs1LG/SfdYsrwgi4p/A+2tYpTdwRyTPA20ktSuBuIouImZGxNjC+w+BSaw+bnTR91cj4yq6wj5YVJhsWXjVbXGRx/5qTFy5kNQB6AXc0sAqufw9NiKuUtak+6xZJoJGaA9MqzU9nRI4yBTsXri8f0TSjsXcsKTOQFfS2WRtue6vNcQFOeyvwu2EccBs4PGIKIn91Yi4IJ/v13XAOcDKBpbn9f26jjXHBfn9PQbwmKQxkvrWs7xJ91mlJoL6BswphbOnsaR6IF2AQcCDxdqwpM1J40ufFhEL6y6u558UZX+tJa5c9ldErIiIXUhjbO8qaac6q+SyvxoRV9H3l6RDgdkRMWZNq9UzL9P91ci4cvt7BPaMiG5AD+BkSfvUWd6k+6xSE8F0oGOt6Q7AjJxi+f8iYuGqy/uIGAm0lNQ26+1Kakk62A6LiPvrWSWX/bW2uPLaX7W2Px8YDRxSZ1Gu36+G4sppf+0JHCbpXeBPwAGS7qyzTh77a61x5fn9iogZhZ+zgQeAXeus0qT7rFITwcPAsYUn792BBRExM++gJG0pSYX3u5L+f+ZlvE0BtwKTIuKaBlYr+v5qTFw57a8qSW0K7zcFDgQm11ktj/211rjy2F8RcV5EdIiIzsCRwBMRcXSd1Yq+vxoTVx77q7CtzSS1XvUeOBio29KwSfdZloPX50bS3aQn/m0lTQcuIj08IyIGAyNJT92nAIuB40okrsOBEyUtB5YAR0ahiUCG9gSOAV4t3F8GOB/YqlZceeyvxsSVx/5qBwyV1IJ0YBgeESMk9asVVx77qzFx5bG/6lUC+6sxceW1vz4PPFDIQRsCd0XEo1nuM5eYMDOrcJV6a8jMzAqcCMzMKpwTgZlZhXMiMDOrcE4EZmYVzonAbD0oVUl9R9JnCtOfLkx3yjs2s8ZyIjBbDxExDbgZGFiYNRAYEhFT84vKbN24H4HZeiqUwhgD3Ab8BOgaER/nG5VZ4zXLnsVmxRQRyySdDTwKHOwkYOXGt4bMmkYPYCZQt+KnWclzIjBbT5J2AQ4ijRR1+voMEGKWBycCs/VQqE55M2m8hPeAK4Gr8o3KbN04EZitn58A70XE44Xpm4DtJe2bY0xm68SthszMKpyvCMzMKpwTgZlZhXMiMDOrcE4EZmYVzonAzKzCORGYmVU4JwIzswr3/wDvQGQVKb6w6QAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt \n",
    "\n",
    "# Plot the data and the linear regression line\n",
    "plt.scatter(X, y, color='blue')\n",
    "plt.plot(X, y_pred, color='red')\n",
    "plt.xlabel('X')\n",
    "plt.ylabel('y')\n",
    "plt.title('Linear Regression')\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
